Topic classification and verification modeling for out-of-domain utterance detection

The detection and handling of OOD (out-of-domain) user utterances are significant problems for spoken language systems. We approach these problems by applying an OOD detection framework, combining topic classification and in-domain verification. In this paper, we compare the performance of three topic classification modeling schemes: 1-vs-all, where a single classifier is trained for each topic; weighted 1-vs-all; and 1-vs-1, which combines multiple pair-wise classifiers. We also compare the performance of a linear discriminate verifier and nonlinear SVM-based verification. In an OOD detection task as a front-end for speech-to-speech translation, detection performance was comparable for all classification schemes, indicating that the simplest 1-vs-all approach is sufficient for this task. SVM-based in-domain verification was found to provide a significant reduction in detection errors compared to a linear discriminate model. However, when the training and testing scenarios differ, the SVM approach was not robust, while the linear discriminate model remained effective.